Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees
نویسندگان
چکیده
Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifically, we approach random Fourier features from a spectral matrix approximation point of view, give tight bounds on the number of Fourier features required to achieve a spectral approximation, and show how spectral matrix approximation bounds imply statistical guarantees for kernel ridge regression.
منابع مشابه
The Error Probability of Random Fourier Features is Dimensionality Independent
We show that the error probability of reconstructing kernel matrices from Random Fourier Features for any shift-invariant kernel function is at most O(exp(−D)), where D is the number of random features. We also provide a matching informationtheoretic method-independent lower bound of Ω(exp(−D)) for standard Gaussian distributions. Compared to prior work, we are the first to show that the error ...
متن کاملFaster Kernel Ridge Regression Using Sketching and Preconditioning
Kernel Ridge Regression (KRR) is a simple yet powerful technique for non-parametric regression whose computation amounts to solving a linear system. This system is usually dense and highly illconditioned. In addition, the dimensions of the matrix are the same as the number of data points, so direct methods are unrealistic for large-scale datasets. In this paper, we propose a preconditioning tec...
متن کاملProvably Useful Kernel Matrix Approximation in Linear Time
We give the first algorithm for kernel Nyström approximation that runs in linear time in the number of training points and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of s landmark points sampled by their ridge leverage scores, requiring just O(ns) kernel evaluations and O(ns) additional r...
متن کاملRecursive Sampling for the Nystrom Method
We give the first algorithm for kernel Nyström approximation that runs in linear time in the number of training points and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of s landmark points sampled by their ridge leverage scores, requiring just O(ns) kernel evaluations and O(ns) additional r...
متن کاملData Dependent Kernel Approximation using Pseudo Random Fourier Features
Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (RFF) has been proposed to scale the kernel method for solving large scale data...
متن کامل